TEXT GENERATION TASKS

TextBox: A Unified, Modularized, and Extensible Framework for Text Generation (2021.acl-demo)

Challenge:	Large language models (LLMs) produce hallucinations, which undermine user trust and reliability.
Approach:	This tutorial offers the first systematic introduction to uncertainty quantification (UQ) for LLMs in text generation tasks.
Outcome:	The proposed framework provides tools for communicating the reliability of a model answer.

Copied to clipboard

Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu, Wayne Xin Zhao, Ji-Rong Wen

ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation (2023.findings-eacl)

Challenge:	TextBox is an open-source text generation framework that is modularized and extensible.
Approach:	They propose to provide a unified, modularized, and extensible text generation framework that implements 21 text generation models on 9 benchmark datasets.
Outcome:	The proposed framework implements 21 models on 9 benchmark datasets and is available under the Apache License 2.0 license.

Copied to clipboard

Wanrong Zhu, Xin Wang, An Yan, Miguel Eckstein, William Yang Wang

Multimodality

Challenge:	Existing evaluation methods for natural language generation rely on token-level or embedding-level comparisons with text references.
Approach:	They propose to use text-to-image generator to generate an image as the embodied imagination for the text snippet and compute the imagination similarity using contextual embeddings.
Outcome:	The proposed metric improves existing evaluation metrics’ correlations with human similarity judgments in both reference-based and reference-free scenarios.

fairseq: A Fast, Extensible Toolkit for Sequence Modeling (N19-4)

Copied to clipboard

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli

Challenge:	OpenNMT is a community-built toolkit written in multiple languages with an emphasis on extensibility.
Approach:	They propose to use PyTorch to train custom sequence models for translation, summarization, language modeling, and other tasks.
Outcome:	The proposed toolkit is fast, extensible, and useful for both research and production.

Controlled Text Generation Using Dictionary Prior in Variational Autoencoders (2022.findings-acl)

Copied to clipboard

Xianghong Fang, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Dit-Yan Yeung

How Does In-Context Learning Help Prompt Tuning? (2024.findings-eacl)

Challenge:	Variational autoencoders (VAEs) have been widely applied in text generation tasks, but they suffer from insufficient representation capacity and poor controllability.
Approach:	They propose a data-driven prior that has expressivity and controllability.
Outcome:	The proposed prior enjoys expressivity and controllability and can be used in language modeling and controlled text generation.

Copied to clipboard

Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer

PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from related Example Banks (2025.naacl-long)

Challenge:	a growing number of parameter-efficient adaptation methods are needed to fine-tune large language models.
Approach:	They propose a method that combines prompt tuning and in-context learning to improve prompt tuning by concatenating a natural language demonstration with learned prompt embeddings.
Outcome:	The proposed method outperforms prompt tuning and prompt tuning on five language generation tasks.

Copied to clipboard

Soumya Suvra Ghosal, Soumyabrata Pal, Koyel Mukherjee, Dinesh Manocha

Low-Resource NLP

Challenge:	Large Language Models (LLMs) have demonstrated impressive few-shot learning capabilities through in-context learning.
Approach:	They propose a novel Alternating Minimization approach for example selection that improves ICL performance on low-resource Indic languages.
Outcome:	The proposed approach outperforms existing frameworks for retrieving examples on low-resource Indic languages.

Non-Autoregressive Text Generation with Pre-trained Language Models (2021.eacl-main)

Copied to clipboard

Yixuan Su, Deng Cai, Yan Wang, David Vandyke, Simon Baker, Piji Li, Nigel Collier

Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach (2024.findings-eacl)

Challenge:	Autoregressive generation models generate tokens in a left-to-right, token-by-token fashion, resulting in lag in inference.
Approach:	They propose to use BERT as the backbone of a non-autoregressive generation model for greatly improved performance.
Outcome:	The proposed model outperforms existing non-autoregressive models and achieves competitive performance with many strong autoregressive model.

Copied to clipboard

Zhuang Li, Levon Haroutunian, Raj Tumuluri, Philip Cohen, Reza Haf

Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling (N19-1)

Challenge:	Large pre-trained language models such as GPT-3.5 and GPT-4 have gained significant attention in natural language research due to limited computational resources or inaccessible parameters.
Approach:	They propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs while editing their output.
Outcome:	The proposed framework significantly improves GPT-3.5’s performance in logical form-to-text conversion and low-resource machine translation, surpassing other state-of-the-art (SOTA) LLM post-editing methods in cross-domain settings.

Copied to clipboard

Prince Zizhuang Wang, William Yang Wang

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)

Challenge:	Empirical experiments show that our model learns latent distributions that respect latent space geometry and is able to generate sentences that are more diverse.
Approach:	They propose a Variational Wasserstein Autoencoder with Riemannian Normalizing Flow to solve this problem by transforming a latent variable into a space that respects the geometric characteristics of input space.
Outcome:	Empirical results show that the proposed model avoids KLvanishing and has better performance in language modeling, likelihood approximation, and text generation tasks.

Copied to clipboard

Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing

Mapping the Design Space of Human-AI Interaction in Text Summarization (2022.naacl-main)

Challenge:	Texar is an open-source text generation toolkit that supports a broad set of text generation tasks.
Approach:	They introduce Texar, an open-source text generation toolkit that supports text generation tasks.
Outcome:	Texar supports machine translation, summarization, dialog, content manipulation, and more.

Copied to clipboard

Ruijia Cheng, Alison Smith-Renner, Ke Zhang, Joel Tetreault, Alejandro Jaimes-Larrarte

Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsing (2023.tacl-1)

Challenge:	Automated text summarization systems involve humans for preparing data or evaluating model performance, yet, there is no systematic understanding of human-AI interactions and how to design for them.
Approach:	They conducted a systematic literature review of 70 papers and designed prototypes for each interaction.
Outcome:	The proposed design considerations were based on the results of a systematic literature review of 70 papers and interviews with 16 users.

Copied to clipboard

Han He, Jinho D. Choi

Syntactic Text Processing

Challenge:	Sequence-to-Sequence (S2S) models have been successful on text generation tasks . however, learning complex structures with S2S models remains challenging .
Approach:	They propose to use constrained decoding to model part-of-speech tagging, named entity recognition, constituency, and dependency parsing tasks with 3 lexically diverse linearization schemas and corresponding constrained coding methods.
Outcome:	The proposed methods outperform the state-of-the-art on four core tasks.

An Exploration of Post-Editing Effectiveness in Text Summarization (2022.naacl-main)

Copied to clipboard

Vivian Lai, Alison Smith-Renner, Ke Zhang, Ruijia Cheng, Wenjuan Zhang, Joel Tetreault, Alejandro Jaimes-Larrarte

mbrs: A Library for Minimum Bayes Risk Decoding (2024.emnlp-demo)

Challenge:	Automated summarization methods are efficient but can suffer from low quality.
Approach:	They conducted an experiment with 72 participants to compare post-editing provided summaries with manual summarization for summary quality, human efficiency, and user experience.
Outcome:	The results show that post-editing improves summary quality, human efficiency, and user experience on formal (XSum news) and informal (Reddit posts) text.

Copied to clipboard

Hiroyuki Deguchi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe

Challenge:	Minimum Bayes risk (MBRS) decoding is a decision rule of text generation tasks that outperforms conventional maximum a posteriori (MAP) decoders by selecting high-quality outputs based on quality or preference rather than probability.
Approach:	They propose to use minimum bayes risk (MBRS) decoding to determine outputs based on quality rather than probability.
Outcome:	MBRS is an MIT-licensed open-source project with a focus on speed, reproducibility, and extensibility.

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models (2024.acl-demos)

Copied to clipboard

Yaowei Zheng, Richong Zhang, Junhao Zhang, YeYanhan YeYanhan, Zheyan Luo

KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation (2026.findings-eacl)

Challenge:	Efficient fine-tuning of large language models requires non-trivial efforts to implement these methods on different models.
Approach:	They propose a framework that democratizes the fine-tuning of large language models by integrating a suite of efficient training methods into one framework.
Outcome:	The proposed framework is able to scale to 100+ LLMs without coding and receives over 25,000 stars and 3,000 forks.

Copied to clipboard

Jiabin Fan, Guoqing Luo, Michael Bowling, Lili Mou

LM-Polygraph: Uncertainty Estimation for Language Models (2023.emnlp-demo)

Challenge:	Empirical evaluation shows that our approach yields superior performance in both standard task metrics and large language model (LLM)-based evaluation.
Approach:	They propose a K-step return estimation method for reinforcement learning (RL)-based knowledge distillation in text generation tasks using the Bellman Optimality Equation.
Outcome:	The proposed method performs better on standard task metrics and large language model evaluations on three text generation tasks.

Copied to clipboard

Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Artem Shelmanov

Repetition Neurons: How Do Language Models Produce Repetitions? (2025.naacl-short)

Challenge:	Large language models often "hallucinate" i.e., fabricate facts without providing users an apparent means to discern the veracity of their statements.
Approach:	They propose a framework with implementations of state-of-the-art UE methods for LLMs with unified program interfaces in Python.
Outcome:	The proposed framework implements state-of-the-art UE methods for LLMs with unified program interfaces in Python and an extendable benchmark for consistent evaluation by researchers.

Copied to clipboard

Tatsuya Hiraoka, Kentaro Inui

TextBox 2.0: A Text Generation Library with Pre-trained Language Models (2022.emnlp-demos)

Challenge:	Existing studies on text generation with LLMs focus on attention heads, framing repetition as a key mechanism for in-context learning.
Approach:	They introduce repetition neurons that are regarded as "skill neurons" responsible for the repetition problem in text generation tasks.
Outcome:	The authors identify repetition neurons as "skill neurons" that perceive repetition as a task to copy the previous context repeatedly, similar to in-context learning.

Copied to clipboard

Tianyi Tang, Junyi Li, Zhipeng Chen, Yiwen Hu, Zhuohao Yu, Wenxun Dai, Wayne Xin Zhao, Jian-yun Nie, Ji-rong Wen

VizSeq: a visual analysis toolkit for text generation tasks (D19-3)

Challenge:	TextBox 2.0 focuses on the use of pre-trained language models (PLMs) to generate text.
Approach:	They propose a library that integrates pre-trained language models into 13 common text generation tasks and 83 datasets.
Outcome:	The proposed library covers 13 common text generation tasks and their corresponding datasets and incorporates 45 PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLM.

Copied to clipboard

Changhan Wang, Anirudh Jain, Danlu Chen, Jiatao Gu

Visual Data in NLP

Challenge:	Several softwares for text evaluation are available that do not provide detailed examples.
Approach:	They propose a visual analysis toolkit for instance-level and corpus-level system evaluation on a wide variety of text generation tasks.
Outcome:	The proposed toolkit covers most common n-gram metrics and latest embedding-based metrics such as BERTScore.

Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation (2022.naacl-main)

Copied to clipboard

Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, Xing Xie

MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance (D19-1)

Challenge:	Variational Auto-Encoders are often used for text generation tasks due to the sequential nature of the text.
Approach:	They propose a variational Transformer framework that learns a series of layer-wise latent variables with each inferred from those of lower layers and tightly coupled with the hidden states by low-rank tensor product.
Outcome:	The proposed framework can learn latent variables from lower layers and incorporate more information.

Copied to clipboard

Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, Steffen Eger

Representation Learning

Challenge:	Existing evaluation metrics are not capable of evaluating text quality.
Approach:	They propose a metric that compares system output against reference texts based on semantics rather than surface forms.
Outcome:	The proposed metric shows a high correlation with human judgment of text quality on a number of text generation tasks.

Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation (2023.tacl-1)

Copied to clipboard

Fei Huang, Pei Ke, Minlie Huang

Select and Attend: Towards Controllable Content Selection in Text Generation (D19-1)

Challenge:	Existing non-AutoRegressive (NAR) text generation models lack proper pre-training, making them far behind pre-trained autoregressive models.
Approach:	They propose a novel pre-training task to promote prediction consistency in non-autoregressive (NAR) generation.
Outcome:	The proposed model outperforms existing pre-trained models and achieves 17 times speedup in throughput.

Copied to clipboard

Xiaoyu Shen, Jun Suzuki, Kentaro Inui, Hui Su, Dietrich Klakow, Satoshi Sekine

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph (2020.emnlp-main)

Challenge:	Recent neural network models conflate content selection and surface realization into a black-box architecture, resulting in content to be described in text cannot be explicitly controlled.
Approach:	They propose to decouple content selection from the decoder to allow finer-grained control over the generation.
Outcome:	The proposed model can be trained end-to-end without human annotations and achieves promising results in data-totext and headline generation tasks.

Copied to clipboard

Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang

ATGen: A Framework for Active Text Generation (2025.acl-demo)

Challenge:	Existing approaches that integrate commonsense knowledge into pre-trained language models simply transfer relational knowledge while ignoring rich connections within the knowledge graph.
Approach:	They propose a method that leverages structural and semantic information of the knowledge graph to generate commonsense-aware text.
Outcome:	The proposed method outperforms baseline models on three text generation tasks that require reasoning over commonsense knowledge.

Copied to clipboard

Akim Tsvigun, Daniil Vasilev, Ivan Tsvigun, Ivan Lysenko, Talgat Bektleuov, Aleksandr Medvedev, Uliana Vinogradova, Nikita Severin, Mikhail Mozikov, Andrey Savchenko, Ilya Makarov, Grigorev Rostislav, Ramil Kuleev, Fedor Zhdanov, Artem Shelmanov

ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation (2022.emnlp-main)

Challenge:	Despite the surging popularity of natural language generation tasks, the application of active learning (AL) to NLG has been limited.
Approach:	They propose a framework that bridges AL with text generation tasks and provides a unified platform for smooth implementation and benchmarking of novel AL strategies tailored to NLG tasks.
Outcome:	The proposed framework simplifies AL-empowered annotation in NLG tasks using both human annotators and automatic annotation agents based on large language models (LLMs).

Copied to clipboard

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Can Large Language Models Generate High-quality Patent Claims? (2025.findings-naacl)

Challenge:	Existing methods for text generation use auto-regressive (AR) methods, but inefficient inference is a problem.
Approach:	They propose an efficient and effective PLM to explicitly model the token dependency during NAR text generation.
Outcome:	The proposed model outperforms existing models on three text generation tasks while achieving 10 times faster inference speedup.

Copied to clipboard

Lekang Jiang, Caiqi Zhang, Pascal A. Scherz, Stefan Goetz

An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation (D18-1)

Challenge:	Large language models (LLMs) have shown exceptional performance across various text generation tasks, but remain under-explored in the patent domain, which offers highly structured and precise language.
Approach:	They construct a dataset to investigate the performance of current LLMs in patent claim generation.
Outcome:	The proposed model outperforms state-of-the-art general LLMs in patent claim generation.

Copied to clipboard

Liangchen Luo, Jingjing Xu, Junyang Lin, Qi Zeng, Xu Sun

Dialogue Systems & Conversational Agents

Challenge:	Experimental results show that our model can generate semantically coherent responses compared to baseline models.
Approach:	They propose an Auto-Encoder Matching model to learn utterance-level semantic dependency . their model contains two auto-encoders and one mapping module .
Outcome:	Experimental results show that the proposed model can generate high coherence and fluency compared to baseline models.

In-Context Demonstration Selection with Cross Entropy Difference (2023.findings-emnlp)

Copied to clipboard

Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu

UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation (2024.emnlp-main)

Challenge:	Large language models (LLMs) can use in-context demonstrations to improve performance on zero-shot tasks.
Approach:	They propose a cross-entropy difference method for selecting in-context demonstrations that uses parameter efficient finetuning to train small models on training data.
Outcome:	The proposed method outperforms baseline selection methods on a mix-domain dataset and shows that the effectiveness of in-context demonstrations negatively correlates with the perplexity of the test example.

Copied to clipboard

Xiangyu Zhao, Yuehan Zhang, Wenlong Zhang, Xiao-Ming Wu

Efficient Out-of-Domain Detection for Sequence to Sequence Models (2023.findings-acl)

Challenge:	e-commerce tasks such as multimodal retrieval and multimodal generation are largely ignored due to the diversity of the multimodal fashion domain.
Approach:	They propose a framework that integrates image generation with retrieval and text generation tasks.
Outcome:	The proposed framework outperforms state-of-the-art models across fashion tasks.

Copied to clipboard

Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, Artem Shelmanov

Towards More Efficient Insertion Transformer with Fractional Positional Encoding (2023.eacl-main)

Challenge:	Sequence-to-sequence (seq2sequ) models are a ubiquitous tool for text generation but they are not suitable for many other tasks.
Approach:	They propose to use UE techniques to identify out-of-domain (OOD) inputs where the model is susceptible to errors.
Outcome:	The proposed methods outperform heavyweight ensembles on the task of OOD detection.

Copied to clipboard

Zhisong Zhang, Yizhe Zhang, Bill Dolan

TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge (2022.findings-acl)

Challenge:	Empirical studies on text generation tasks demonstrate the effectiveness of insertion-based models.
Approach:	They propose a reusable positional encoding scheme for insertion transformers that allows reusing representations calculated in previous steps.
Outcome:	Empirical studies show that the proposed model reduces the time required to generate a token and improves decoding efficiency.

Copied to clipboard

Chao-Hong Tan, Jia-Chen Gu, Chongyang Tao, Zhen-Hua Ling, Can Xu, Huang Hu, Xiubo Geng, Daxin Jiang

Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR (2021.eacl-main)

Challenge:	Generating natural and informative texts has been a long-standing problem in NLP.
Approach:	They propose to augment TExt Generation via Task-specific and Open-world Knowledge in a unified framework.
Outcome:	The proposed model can learn what and how to generate on two text generation tasks.

Copied to clipboard

Juri Opitz, Anette Frank

Exploring and Adapting Chinese GPT to Pinyin Input Method (2022.acl-long)

Challenge:	Abstract meaning representations are typically evaluated using surface matching metrics . however, there are problems with these metrics, since they allow multiple surface realizations .
Approach:	They propose a decomposable metric that measures the distance between the original and reconstructed AMRs.
Outcome:	The proposed metric reduces the complexity of the evaluation process by allowing for multiple surface realizations.

Copied to clipboard

Minghuan Tan, Yong Dai, Duyu Tang, Zhangyin Feng, Guoping Huang, Jing Jiang, Jiwei Li, Shuming Shi

What’s Wrong? Refining Meeting Summaries with LLM Feedback (2025.coling-main)

Challenge:	a frozen GPT can generate state-of-the-art performance on perfect pinyin, but performance drops when input includes abbreviated pinyan, which links to even larger number of Chinese characters.
Approach:	They propose to use Chinese GPT to generate fluent sentences using abbreviated pinyin.
Outcome:	The proposed approach improves on abbreviated pinyin across all domains.

Copied to clipboard

Frederic Thomas Kirstein, Terry Lima Ruas, Bela Gipp

Summarization

Challenge:	Existing methods for meeting summarization are limited and lack the robustness and context-based accuracy needed to maintain relevance.
Approach:	They propose a multi-LLM correction approach for meeting summarization using a two-phase process that mimics the human review process: mistake identification and summary refinement.
Outcome:	The proposed approach improves the quality of a given meeting summarization measured by relevance, informativeness, conciseness, and coherence.

SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models (2024.eacl-long)

Copied to clipboard

Xiang Gao, Jiaxin Zhang, Lalla Mouatadid, Kamalika Das

Dynamic and Efficient Inference for Text Generation via BERT Family (2023.acl-long)

Challenge:	Large language models have a tendency to make confidently wrong predictions, highlighting the need for uncertainty quantification (UQ) . previous studies focused on aleatoric uncertainty, but the full spectrum of uncertainties, including epistemic, remains inadequately explored.
Approach:	They propose a method to quantify uncertainty in large language models (LLMs) they use a set of perturbations and an aggregation module to generalize the method.
Outcome:	The proposed method improves model uncertainty calibration and reduces expected calibration error by 50% on average.

Copied to clipboard

Xiaobo Liang, Juntao Li, Lijun Wu, Ziqiang Cao, Min Zhang

PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation (2022.acl-long)

Challenge:	Existing methods to improve performance of pre-trained language models are limited due to large-scale parameters and the universal autoregressive decoding paradigm.
Approach:	They propose a novel fine-tuning method which can make a single pre-trained model support Dynamic and Efficient infERence and achieve an adaptive trade-off between model performance and latency.
Outcome:	The proposed method achieves higher BLEU scores than the strong autoregressive Transformer model on translation tasks with 3 12 times speedup and faster inference speed compared with the BART model on four GLGE benchmark tasks.

Copied to clipboard

Zhe Hu, Hou Pong Chan, Jiachen Liu, Xinyan Xiao, Hua Wu, Lifu Huang

Data Augmentation for Text Generation Without Any Augmented Data (2021.acl-long)

Challenge:	Existing methods for text generation still suffer from incoherence problems . Neural sequence-to-sequence (seq2sequ) models generate fluent results .
Approach:	They propose a novel generation framework that leverages autoregressive self-attention mechanism to conduct content planning and surface realization dynamically.
Outcome:	The proposed framework outperforms baseline models and generates more coherent texts with richer contents.

Copied to clipboard

Wei Bi, Huayang Li, Jiacheng Huang

A Distributional and Orthographic Aggregation Model for English Derivational Morphology (P18-1)

Challenge:	Existing methods for data augmentation need to define or choose proper data mapping functions to create augmented samples.
Approach:	They propose to use data mapping functions to augment text samples without using specific mapping functions.
Outcome:	The proposed approach can approximate or even surpass popular data augmentation methods on two text generation tasks with a convergence rate guarantee.

Copied to clipboard

Daniel Deutsch, John Hewitt, Dan Roth

Morphology

Challenge:	Existing approaches to derived word generation model derivational morphology to generate words with particular semantics are not effective.
Approach:	They propose a novel aggregation model that learns derivational transformations as orthographic functions and as functions in distributional word embedding space.
Outcome:	The proposed model learns to choose between the hypothesis of each system and the hypothesis from the model.

B2T Connection: Serving Stability and Performance in Deep Transformers (2023.findings-acl)

Copied to clipboard

Sho Takase, Shun Kiyono, Sosuke Kobayashi, Jun Suzuki

Sequence-level Large Language Model Training with Contrastive Preference Optimization (2025.findings-naacl)

Challenge:	Existing methods to prevent the vanishing gradient problem in deep neural networks are not effective.
Approach:	They propose a method that can equip both higher stability and effective training by a simple modification from Post-LN.
Outcome:	The proposed method outperforms Pre-LN and Post-Ln on a wide range of tasks.

Copied to clipboard

Zhili Feng, Dhananjay Ram, Cole Hawkins, Aditya Rawal, Jinman Zhao, Sheng Zha

Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity (2025.findings-acl)

Challenge:	a new method to improve the performance of large language models requires a small computational cost.
Approach:	They propose a CPO procedure that can inject sequence-level information into the model at any training stage without expensive human labeled data.
Outcome:	The proposed objective surpasses the next token prediction in terms of win rate in instruction-following and text generation tasks.

Copied to clipboard

Dang Nguyen, Ali Payani, Baharan Mirzasoleiman

Text Clustering

Challenge:	Large Language Models (LLMs) generate long one-sentence responses that are less effective because they overlook two crucial factors: intra-cluster similarity and inter-c cluster similarity.
Approach:	They propose a method that generalizes semantic entropy and uses token probabilities to quantify uncertainty in large language models.
Outcome:	The proposed method can be extended to white-box settings by incorporating token probabilities.

Who Wrote This Line? Evaluating the Detection of LLM-Generated Classical Chinese Poetry (2026.acl-long)

Copied to clipboard

Jiang Li, Tian Lan, Shanshan Wang, Dongxing Zhang, Dianqing Lin, Guanglai Gao, Derek F. Wong, Xiangdong Su

Challenge:	a recent study shows that large language models can generate text, but they can also fabricate large amounts of false or misleading content.
Approach:	They propose a benchmark to detect LLM-generated classical Chinese poetry . they compare 12 different AI detectors to find out whether a poem is authored by AI .
Outcome:	The proposed benchmark compared 12 AI detectors with a dataset of 30,664 Chinese poems . the results highlight the limitations of current Chinese text detectors .

Learning to Transfer Prompts for Text Generation (2022.naacl-main)

Copied to clipboard

Junyi Li, Tianyi Tang, Jian-Yun Nie, Ji-Rong Wen, Xin Zhao

Empowering Diffusion Models on the Embedding Space for Text Generation (2024.naacl-long)

Challenge:	Pretrained language models (PLMs) have made remarkable progress in text generation tasks via fine-tuning.
Approach:	They propose a prompt-based method that learns source prompts and transfers them as target prompts to perform target generation tasks.
Outcome:	The proposed method can be used to perform text generation tasks in a transferable setting.

Copied to clipboard

Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, Linli Xu

Representation Learning

Challenge:	Recent work adapts diffusion models to textual data by diffusing on the embedding space.
Approach:	They propose an embedding diffusion model based on Transformer to solve the problem of embeddable space and denoising model.
Outcome:	The proposed model is more efficient than previous methods on seminal text generation tasks and is superior to existing models.

Aligning as Debiasing: Causality-Aware Alignment via Reinforcement Learning with Interventional Feedback (2024.naacl-long)

Copied to clipboard

Yu Xia, Tong Yu, Zhankui He, Handong Zhao, Julian McAuley, Shuai Li

SyntaxShap: Syntax-aware Explainability Method for Text Generation (2024.findings-acl)

Challenge:	Existing methods to reduce LLMs' biased outputs rely on reward signals from current model outputs without considering the source of biases.
Approach:	They propose to leverage the reward model in RL alignment as an instrumental variable to perform causal intervention on LLMs.
Outcome:	The proposed method reduces biases by using human feedback to fine tune LLMs to human values.

Copied to clipboard

Kenza Amara, Rita Sevastjanova, Mennatallah El-Assady

Explainability & Interpretability in NLP

Challenge:	Existing methods for explaining sequence-to-sequence tasks are not suitable for textual data.
Approach:	They propose a local, model-agnostic explainability method that takes into account the syntax in text data and extends Shapley values to account for parsing-based syntactic dependencies.
Outcome:	The proposed method is compared to state-of-the-art explainability methods for text generation tasks using various metrics including faithfulness, coherency, and semantic alignment of the explanations to the model.

SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes (2023.acl-long)

Copied to clipboard

Wenda Xu, Xian Qian, Mingxuan Wang, Lei Li, William Yang Wang

Principled Self-Correction in Discrete Diffusion: A UCB-Guided Framework for Text Generation (2026.eacl-long)

Challenge:	Existing learned metrics perform unsatisfactory across text generation tasks or require human annotations for training on specific tasks.
Approach:	They propose a self-supervised approach to train a model-based metric for text generation evaluation using sentences retrieved from a corpus.
Outcome:	The proposed model outperforms all prior unsupervised metrics on four text generation evaluation benchmarks, with an average Kendall improvement of 0.158.

Copied to clipboard

Masaki Asada, Makoto Miwa

Do you have the right scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods (2020.acl-main)

Challenge:	Existing diffusion models are trained on corrupted ground-truth tokens, but at inference time they must denoise inputs corruptes from their own predictions.
Approach:	They propose a framework that denoises inputs corrupted from their own predictions at inference time.
Outcome:	The proposed framework achieves higher faithfulness and coherence over existing diffusion baselines.

Copied to clipboard

Ning Miao, Yuxuan Song, Hao Zhou, Lei Li

Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment (2025.coling-main)

Challenge:	Pre-trained language models can be fine-tuned on task-specific datasets, but fine-timing can lead to over- and/or under-estimation problems.
Approach:	They propose a method to transfer probability mass from over-estimated regions to under-estimates by truncating and transferring probability mass between over- and under-estimating regions.
Outcome:	The proposed method outperforms the fine-tuning approach on a variety of datasets.

Copied to clipboard

Jianfei Zhang, Jun Bai, Bei Li, Yanmeng Wang, Rumei Li, Chenghua Lin, Wenge Rong

Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning (2025.findings-naacl)

Challenge:	Human values are inherently diverse, making it insufficient to align LLMs solely with general preferences.
Approach:	They propose a flexible paradigm for individual preference alignment that disentangles preference representation from text generation in LLMs.
Outcome:	The proposed method produces aligned quality and better than PEFT-based methods while reducing training time for each new individual preference by 80% to 90%.

Copied to clipboard

Hyundong Justin Cho, Karishma Sharma, Nicolaas Paul Jedema, Leonardo F. R. Ribeiro, Jonathan May, Alessandro Moschitti

Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization (2021.emnlp-main)

Challenge:	Language models are biased towards generic outputs as they are trained to align to an aggregate preference to be generally useful.
Approach:	They propose a tuning-free method that personalizes language models for text generation tasks with fewer than 10 examples per user.
Outcome:	The proposed method achieves favorable win rates on pairwise comparisons with the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles.

Copied to clipboard

Tiezheng Yu, Wenliang Dai, Zihan Liu, Pascale Fung

Multimodality

Challenge:	Existing MAS models cannot leverage GPLMs’ powerful generation ability.
Approach:	They propose a method to construct vision guided (VG) GPLMs that incorporate visual information while maintaining their original text generation ability.
Outcome:	The proposed model outperforms the previous state-of-the-art model by 5.7 ROUGE-1, 5.3 ROUGe-2, and 5.1 ROUGEL-L scores on the How2 dataset and contributes 83.6% of the overall improvement.

GPTScore: Evaluate as You Desire (2024.naacl-long)

Copied to clipboard

Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, Pengfei Liu

Hurdles to Progress in Long-form Question Answering (2021.naacl-main)

Challenge:	Existing evaluation frameworks for text generation are not adequate to assess the quality of the generated outputs.
Approach:	They propose a framework that utilizes emergent abilities of generative pre-trained models to evaluate generated texts.
Outcome:	The proposed evaluation framework can achieve what one desires to evaluate for texts simply by natural language instructions.

Copied to clipboard

Kalpesh Krishna, Aurko Roy, Mohit Iyyer

Information Retrieval

Challenge:	Long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer.
Approach:	They propose a long-form question answering system that relies on sparse attention and contrastive retriever learning to achieve state-of-the-art performance on the ELI5 LFQA dataset.
Outcome:	The proposed system tops the public leaderboard on the ELI5 LFQA dataset, but it has several troubling issues.

LaMP: When Large Language Models Meet Personalization (2024.acl-long)

Copied to clipboard

Alireza Salemi, Sheshera Mysore, Michael Bendersky, Hamed Zamani

Correction of Errors in Preference Ratings from Automated Metrics for Text Generation (2023.findings-acl)

Challenge:	Existing benchmarks for personalization in large language models are understudied .
Approach:	They propose a benchmark for training and evaluating language models for producing personalized outputs using a set of seven personalized tasks . they propose two retrieval augmentation approaches that retrieve personal items from each user profile for personalizing language model outputs.
Outcome:	The proposed approach is effective for a set of zero-shot and fine-tuned language models and highlights the impact of personalization in various natural language tasks.

Copied to clipboard

Jan Deriu, Pius von Däniken, Don Tuggener, Mark Cieliebak

Denoising based Sequence-to-Sequence Pre-training for Text Generation (D19-1)

Challenge:	Existing evaluation methods are over-confident in assigning significant differences between systems . Currently, the most reliable evaluation methods for text generation are human-based evaluations.
Approach:	They propose to combine human ratings with automated ratings to reduce the amount of human ratings needed to arrive at robust results.
Outcome:	The proposed evaluation protocol reduces the amount of human ratings by 50% while yielding the same evaluation outcome as the pure human evaluation in 95% of cases.

Copied to clipboard

Liang Wang, Wei Zhao, Ruoyu Jia, Sujian Li, Jingming Liu

Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning (2023.emnlp-main)

Challenge:	PoDA pre-trains encoders and decoders by denoising noise-corrupted text . Unlike encoder-only or decode-only methods, it can be used for text generation tasks without using any task-specific techniques.
Approach:	They propose a sequence-to-sequence (seq2sequ) pre-training method PoDA which denoises autoencoders by denoising noise-corrupted text.
Outcome:	The proposed method improves model performance over strong baselines without using any task-specific techniques and significantly speed up convergence.

Copied to clipboard

Ximing Lu, Faeze Brahman, Peter West, Jaehun Jung, Khyathi Chandu, Abhilasha Ravichander, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Lin, Skyler Hallinan, Lianhui Qin, Xiang Ren, Sean Welleck, Yejin Choi

Argument Mining with Fine-Tuned Large Language Models (2025.coling-main)

Challenge:	Extreme-scale language models have shown exceptional performance on a variety of language tasks, but the degree of control offered by these models through pure prompting is limited.
Approach:	They propose an inference-time policy adapter which tailors a large base model without fine-tuning it.
Outcome:	The proposed model outperforms baseline methods on five challenging text generation tasks and even over GPT-4.

Copied to clipboard

Jérémie Cabessa, Hugo Hernault, Umer Mushtaq

Generalized Supervised Attention for Text Generation (2021.findings-acl)

Challenge:	Argument Mining (AM) pipelines use fine-tuned large language models (LLMs) . initial approaches employ supervised machine learning algorithms, such as Maximum Entropy classifiers and Logistic Regressions.
Approach:	They propose to model the three main AM sub-tasks as text generation tasks and fine-tune eight popular quantized and non-quantized large language models (LLMs) on the benchmark PE, AbstRCT, and CDCP datasets.
Outcome:	The proposed pipeline achieves state-of-the-art across all AM sub-tasks and datasets, showing significant improvements over previous benchmarks.

Copied to clipboard

Yixian Liu, Liwen Zhang, Xinyu Zhang, Yong Jiang, Yue Zhang, Kewei Tu

White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs (2025.acl-long)

Challenge:	Existing supervised attention methods that use human knowledge to learn better alignments are costly or infeasible.
Approach:	They propose a generalized supervised attention method based on quasi alignments that are easier to obtain than ideal alignments.
Outcome:	The proposed framework improves generation performance and is robust against errors in attention supervision.

Copied to clipboard

Yixin Wan, Kai-Wei Chang

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models (2025.findings-emnlp)

Challenge:	Social biases manifest in language agency, but there is no comprehensive benchmark for evaluating such biase in language models.
Approach:	They propose a benchmark to evaluate language agency biases in large language models . they propose 'Mitigation via Selective Rewrite' to selectively revise parts of generated texts .
Outcome:	The proposed language agency bias evaluation benchmark identifies gender, racial, and intersectional biases in 3 recent LLMs.

Copied to clipboard

Tao Fan, Weijing Chen, Yan Kang, Guoqiang Ma, Hanlin Gu, Yuanfeng Song, Lixin Fan, Qiang Yang

POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)

Challenge:	Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks.
Approach:	They propose a federated framework for the Chain-of-Thought distillation of knowledge from LLMs to SLMs, while adhering to privacy requirements.
Outcome:	The proposed framework ensures secure knowledge transfer from an LLM on a high-powered server to an SLM on resource-constrained client while adhering to privacy requirements.

Copied to clipboard

Kexin Yang, Wenqiang Lei, Dayiheng Liu, Weizhen Qi, Jiancheng Lv

DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off (2025.emnlp-main)

Challenge:	Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding.
Approach:	They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model .
Outcome:	The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation.

Copied to clipboard

Jusheng Zhang, Yijia Fan, Kaitong Cai, Zimeng Huang, Xiaofei Sun, Jian Wang, Chengpei Tang, Keze Wang

Green & Sustainable NLP

Challenge:	et al., 2019; Brown e.t al, 2023; Touvron e t al; 2024; OpenAI, 2024) Large Language Models (LLMs) have demonstrated remarkable capabilities in knowledge encoding and contextual understanding during their pretraining phase.
Approach:	They propose a dynamic expert scheduling mechanism that allocates computational resources based on text complexity and a hierarchical sparse attention mechanism that adjusts attention patterns according to a variety of input lengths.
Outcome:	The proposed framework overpowers existing methods on long-text generation benchmarks.

Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation (2022.emnlp-main)

Copied to clipboard

Wenhao Wu, Wei Li, Jiachen Liu, Xinyan Xiao, Sujian Li, Yajuan Lyu

Joint Generator-Ranker Learning for Natural Language Generation (2023.findings-acl)

Challenge:	Existing models for text generation are weak enough to handle perturbations in inputs, leading to degeneration in faithfulness and informativeness.
Approach:	They propose a framework for improving faithfulness and informativeness of Seq2Seq models by perturbing word representations and word swapping.
Outcome:	The proposed framework improves faithfulness and informativeness of Seq2Seq models under automatic and human evaluation settings.

Copied to clipboard

Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, Weizhu Chen

Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation (D19-1)

Challenge:	Existing methods for text generation train the generator and ranker individually . existing methods neglect the mutual feedback that could enhance the quality of outputs .
Approach:	They propose a joint training algorithm that integrates the generator and ranker in a single framework.
Outcome:	The proposed algorithm surpasses existing methods on four public datasets across three common generation scenarios.

Copied to clipboard

Zhangming Chan, Xiuying Chen, Yongliang Wang, Juntao Li, Zhiqiang Zhang, Kun Gai, Dongyan Zhao, Rui Yan

Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding (2024.findings-acl)

Challenge:	Existing models for product description generation do not take the product attribute information into account.
Approach:	They propose a model that takes the embedding and the entity label of each word into account . they establish a keyword memory that stores the entity labels as keys and keywords as values .
Outcome:	The proposed model increases the fidelity of the generated descriptions by 25%.

Copied to clipboard

Yuu Jinnai, Kaito Ariu

Enhancing Reinforcement Learning with Dense Rewards from Language Model Critic (2024.emnlp-main)

Challenge:	Minimum Bayes-Risk (MBR) decoding is a powerful alternative to beam search for text generation tasks.
Approach:	They propose a method that uses a hyperparameter-free method to run MBR decoding efficiently.
Outcome:	The proposed method reduces the amount of computation but requires hyperparameter tuning to be effective.

Copied to clipboard

Meng Cao, Lei Shu, Lei Yu, Yun Zhu, Nevan Wichers, Yinxiao Liu, Lei Meng

EEL: Efficiently Encoding Lattices for Reranking (2023.acl-long)

Challenge:	Reinforcement learning (RL) can align language models with non-differentiable reward signals, such as human preferences, but the sparsity of these signals can lead to inefficient and unstable learning.
Approach:	They propose a framework that utilizes the critique capability of Large Language Models to produce intermediate-step rewards during RL training.
Outcome:	The proposed framework improves sample efficiency and the overall performance of the policy model, supported by both automatic and human evaluation.

Copied to clipboard

Prasann Singhal, Jiacheng Xu, Xi Ye, Greg Durrett

Efficient (Soft) Q-Learning for Text Generation with Limited Good Data (2022.findings-emnlp)

Challenge:	Existing methods for decoding conditional text are slow to apply to large numbers of hypotheses.
Approach:	They propose a method that can efficiently encode lattices of generated outputs using Transformers.
Outcome:	The proposed method can extract high-quality hypotheses from lattices with minimal degradation error compared to naive reranking methods.

Copied to clipboard

Han Guo, Bowen Tan, Zhengzhong Liu, Eric Xing, Zhiting Hu

Improving NMT Models by Retrofitting Quality Estimators into Trainable Energy Loss (2025.coling-main)

Challenge:	Maximum likelihood estimation (MLE) is the predominant method for training text generation models.
Approach:	They propose a new RL formulation for text generation from the soft Q-learning perspective using path consistency learning to combine the best of on-/off-policy updates and learn effectively from sparse reward.
Outcome:	The proposed approach outperforms MLE and previous RL methods in a wide range of tasks.

Copied to clipboard

Gahyun Yoo, Jay Yoon Lee

Multilinguality

Challenge:	Reinforcement learning has shown great promise in aligning language models with human preferences in a variety of text generation tasks, including machine translation.
Approach:	They propose a method that employs quality estimators as trainable loss networks to backpropagate to the NMT model.
Outcome:	The proposed method outperforms strong baselines and proximal policy optimizations on English-to-Mongolian translation.

Isotropy-Enhanced Conditional Masked Language Models (2023.findings-emnlp)

Copied to clipboard

Pei Guo, Yisheng Xiao, Juntao Li, Yixin Ji, Min Zhang

Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes (2025.acl-long)

Challenge:	Existing non-autoregressive models with auto-regressing decoding paradigms have been used for various text generation tasks to accelerate inference but at the cost of generation quality to some extent.
Approach:	They propose to use Look Neighbors strategy to enhance learning of target token representations during training to achieve a good balance between inference speedup and generation quality.
Outcome:	The proposed models outperform current models on 4 WMT datasets and outperformed the current SoTA results.

Copied to clipboard

Bocheng Li, Zhujin Gao, Linli Xu

Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation (2025.coling-main)

Challenge:	Experimental results demonstrate NeoDiff’s superior performance compared to baselines of non-autoregressive continuous and discrete diffusion models, iterative-based methods and autoregressive diffusion-based approaches.
Approach:	They propose a discrete and continuous diffusion model that integrates the strengths of discrete, continuous and continuous approaches.
Outcome:	The proposed model unifies the theories of discrete and continuous diffusion models, offering a more principled and effective framework for text generation.

Copied to clipboard

Samin Mahdizadeh Sani, Pouya Sadeghi, Thuy-Trang Vu, Yadollah Yaghoobzadeh, Gholamreza Haffari

Information Retrieval

Challenge:	Large language models (LLMs) are mainly trained on English data and struggle with low-resource languages.
Approach:	They propose to add a new language to Llama to improve classification accuracy for Persian tasks by aligning representations through bilingual pretraining and instruction datasets.
Outcome:	The proposed model performs on generation and classification tasks with no adverse impact and sometimes even improvements on English tasks.

Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study (P19-1)

Copied to clipboard

Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)

Challenge:	Sequence-to-sequence (seq2sequ) models have a weakness: they cannot always generate sentences without grammatical errors.
Approach:	They propose to use automatic grammatical error correction to improve seq2seq models . they conduct experiments on machine translation, formality style transfer, sentence compression and simplification .
Outcome:	The proposed system can improve grammaticality of generated text and improve formal style tasks.

Copied to clipboard

Bocheng Li, Zhujin Gao, Yongxin Zhu, Kun Yin, Haoyu Cao, Deqiang Jiang, Linli Xu

Low-Resource NLP

Challenge:	Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates.
Approach:	They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach.
Outcome:	The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps.

Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection (2023.emnlp-main)

Copied to clipboard

Xinlin Peng, Ying Zhou, Ben He, Le Sun, Yingfei Sun

Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference (2023.findings-acl)

Challenge:	Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks, but their utilization carries inherent risks, including plagiarism and the dissemination of fake news.
Approach:	They propose to use a dataset to construct an AI-generated student essay that employs a range of text perturbation methods to evade detection.
Outcome:	The proposed methods evade detection and maintain quality of the generated essays while avoiding plagiarism and fake news.

Copied to clipboard

Wangchunshu Zhou, Ronan Le Bras, Yejin Choi

Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation (2025.coling-main)

Challenge:	Pre-trained sequence-to-sequence models have advanced the state of the art on text generation tasks.
Approach:	They introduce a modular encoder-decoder framework for flexible sequence-to-sequence model compression.
Outcome:	The proposed framework can achieve flexible compression ratios from 1.1x to 6x with little to moderate relative performance drop.

Copied to clipboard

Esteban Garces Arias, Meimingwei Li, Christian Heumann, Matthias Assenmacher

Duwak: Dual Watermarks in Large Language Models (2024.findings-acl)

Challenge:	Generative large language models generate a high-dimensional probability distribution over all tokens in their vocabulary.
Approach:	They conduct extensive sensitivity analyses to determine how hyperparameter choices shape the outputs of generative large language models.
Outcome:	The proposed methods influence the distribution of diversity and coherence metrics in human-written text, but the optimal configurations vary across models and tasks.

Copied to clipboard

Chaoyi Zhu, Jeroen Galjaard, Pin-Yu Chen, Lydia Chen

Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference ! (2023.findings-acl)

Challenge:	Existing watermark techniques are effective in embedding single human-imperceptible and machine-detectable patterns without significantly affecting generated text quality and semantics.
Approach:	They propose to embed dual secret patterns in token probability distribution and sampling schemes to enhance the efficiency of watermarking.
Outcome:	The proposed method achieves highest watermark quality at the lowest required token count for detection, up to 70% less than existing techniques, especially under post paraphrasing attacks.

Copied to clipboard

Zecheng Tang, Pinzheng Wang, Keyan Zhou, Juntao Li, Ziqiang Cao, Min Zhang

Models See Hallucinations: Evaluating the Factuality in Video Captioning (2023.emnlp-main)

Challenge:	Existing models for text generation use a discrete data embedding module to map the data into the continuous space.
Approach:	They propose two methods to bridge the gap between training and inference by mapping the discrete text into the continuous space.
Outcome:	The proposed methods can achieve 100 200 speedup with better performance on 6 generation tasks.

Copied to clipboard

Hui Liu, Xiaojun Wan

Multimodality

Challenge:	Existing evaluation metrics show little correlation with human factuality annotation.
Approach:	They propose a weakly-supervised, model-based factuality metric FactVC which outperforms previous metrics on factual evaluation of video captioning.
Outcome:	The proposed model outperforms previous metrics on factuality evaluation of video captioning.

Improving Text Generation with Student-Forcing Optimal Transport (2020.emnlp-main)

Copied to clipboard

Jianqiao Li, Chunyuan Li, Guoyin Wang, Hao Fu, Yuhchen Lin, Liqun Chen, Yizhe Zhang, Chenyang Tao, Ruiyi Zhang, Wenlin Wang, Dinghan Shen, Qian Yang, Lawrence Carin

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding (2024.emnlp-main)

Challenge:	Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias.
Approach:	They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias.
Outcome:	The proposed method is validated on machine translation, text summarization, and text generation tasks.

Copied to clipboard

Weilin Zhao, Yuxiang Huang, Xu Han, Wang Xu, Chaojun Xiao, Xinrong Zhang, Yewei Fang, Kaihuo Zhang, Zhiyuan Liu, Maosong Sun

Challenge:	Speculative decoding is a widely used method that accelerates the generation process of large language models (LLMs) drafting efficiency has become a bottleneck in the final speedup of speculative drafting, therefore generating longer drafts at less cost can lead to better speedup.
Approach:	They propose a method that uses existing model to drafting and target LLM to verify draft in a low-cost parallel manner.
Outcome:	The proposed method can achieve speedups of up to 2.4 over speculative decoding and 3.9 over vanilla decoding without fine-tuning draft and target models.

LAQuer: Localized Attribution Queries in Content-grounded Generation (2025.acl-long)

Copied to clipboard

Eran Hirsch, Aviv Slobodkin, David Wan, Elias Stengel-Eskin, Mohit Bansal, Ido Dagan

PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation (2025.emnlp-main)

Challenge:	Existing attribution methods associate entire sentences with source documents, which can be overwhelming for users seeking to fact-check specific claims.
Approach:	They propose a task that localizes selected spans of generated output to their corresponding source spans, allowing fine-grained and user-directed attribution.
Outcome:	The proposed task localizes selected spans of generated output to their corresponding source spans, allowing fine-grained and user-directed attribution.

Copied to clipboard

Tao Fan, Guoqiang Ma, Yuanfeng Song, Lixin Fan, Qiang Yang

Improving Copy-oriented Text Generation via EDU Copy Mechanism (2024.lrec-main)

Challenge:	Large Language Models (LLMs) are becoming more popular and are gaining widespread use in artificial intelligence.
Approach:	They propose a unified framework that addresses both privacy preservation and model compression in federated settings.
Outcome:	The proposed framework maintains competitive performance comparable to full-sized LLMs while ensuring robust privacy protection through its federated architecture.

Copied to clipboard

Tianxiang Wu, Han Chen, Luozheng Qin, Ziqiang Cao, Chunhui Ai

Syntactic Text Processing

Challenge:	Existing extractive models generate texts through word-by-word decoding, causing factual inconsistencies and slow inference.
Approach:	They propose a framework that integrates the behavior of copying EDUs into generative models.
Outcome:	The proposed framework reduces the number of generated tokens significantly.

AUTOSUMM: Automatic Model Creation for Text Summarization (2021.emnlp-main)

Copied to clipboard

Sharmila Reddy Nangi, Atharv Tyagi, Jay Mundra, Sagnik Mukherjee, Raj Snehal, Niyati Chhaya, Aparna Garimella

Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction (2025.findings-emnlp)

Challenge:	Recent efforts to develop deep learning models for text generation tasks are challenging for non-experts.
Approach:	They propose methods to automatically create deep learning models for extractive and abstractive summarization tasks using large language models.
Outcome:	The proposed methods achieve near state-of-the-art performance on a range of datasets.

Copied to clipboard

Zhexiong Liu, Diane Litman

Information Retrieval

Challenge:	Large Language Models have shown extraordinary success across text generation tasks . however, their potential for simple yet essential text classification remains underexplored .
Approach:	a plug-and-play layer-wise parameter-efficient fine-tuning framework is proposed . it fine- tunes a subset of important LLM layers while freezing redundant ones .
Outcome:	a plug-and-play framework fine-tunes a subset of important LLM layers while freezing redundant layers.

Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding (2024.emnlp-main)

Copied to clipboard

Lifu Tu, Semih Yavuz, Jin Qu, Jiacheng Xu, Rui Meng, Caiming Xiong, Yingbo Zhou

PIXAR: Auto-Regressive Language Modeling in Pixel Space (2024.findings-acl)

Challenge:	Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest.
Approach:	They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions.
Outcome:	The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering.

Copied to clipboard

Yintao Tai, Xiyang Liao, Alessandro Suglia, Antonio Vergari

Finding Blind Spots in Evaluator LLMs with Interpretable Checklists (2024.emnlp-main)

Challenge:	Recent work shows the possibility of building open-vocabulary large language models that operate on pixel representations.
Approach:	They propose a pixel-based autoregressive LLM that performs generative tasks . they propose an adversarial pretraining stage that improves the readability and accuracy of PIXAR by 8.1 on LAMBADA and 8.5 on bAbI.
Outcome:	The proposed model performs free-form generative tasks while keeping the number of parameters on par with previous encoder-decoder models.

Copied to clipboard

Sumanth Doddapaneni, Mohammed Khan, Sshubam Verma, Mitesh Khapra

Explainability & Interpretability in NLP

Challenge:	Large Language Models are increasingly relied upon to evaluate text outputs of other LLMs . however, concerns persist over the accuracy of these assessments and the potential for misleading conclusions.
Approach:	They propose a framework to assess the reliability of Large Language Models (LLMs) they propose ' FBI' framework to examine the proficiency of Evaluator LLMs in assessing four critical abilities .
Outcome:	The proposed framework assesses the performance of LLMs in text generation tasks.

Extrinsic Evaluation of Cultural Competence in Large Language Models (2024.findings-emnlp)

Copied to clipboard

Shaily Bhatt, Fernando Diaz

SPIDE: Serial and Parallel Intertwined Speculative Decoding (2026.findings-acl)

Challenge:	Prior studies have evaluated models’ knowledge of cultural norms, values, and artefacts, without considering how this knowledge manifests in downstream applications.
Approach:	They propose to evaluate model outputs when an explicit cue of culture, specifically nationality, is perturbed in prompts.
Outcome:	The outputs of two open-ended question answering and story generation tasks are evaluated when an explicit cue of culture, specifically nationality, is perturbed in the prompts.

Copied to clipboard

Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan, Cheng Wang

Green & Sustainable NLP

Challenge:	Speculative decoding (SD) is a training-free SD framework that orchestrates dynamic alternation combining serial dynamic drafting with parallel draft verification.
Approach:	They propose a serial and parallel intertwined speculative DEcoding framework that orchestrates dynamic alternation combining serial dynamic drafting and parallel draft verification.
Outcome:	The proposed framework accelerates inference while reducing the LLM usage costs.

Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion (2025.findings-acl)

Copied to clipboard

Tiehan Cui, Yanxu Mao, Peipei Liu, Congying Liu, Datao You

JoPA: Explaining Large Language Model’s Generation via Joint Prompt Attribution (2025.acl-long)

Challenge:	Existing jailbreak methods face an excessive number of iterative queries and poor generalization across models.
Approach:	They propose a jailbreak method that employs Intent Concealment and divErsion to circumvent security constraints.
Outcome:	The proposed method outperforms existing jailbreak techniques in question-answering and text-generation tasks.

Copied to clipboard

Yurui Chang, Bochuan Cao, Yujia Wang, Jinghui Chen, Lu Lin

QuASAR: A Question-Driven Structure-Aware Approach for Table-to-Text Generation (2025.acl-long)

Challenge:	Existing attempts to explain the entire language generation often treat input prompt texts independently, ignoring their combinatorial effects on the follow-up generation.
Approach:	They propose a framework for explaining how a few prompt texts collaboratively influences the LLM's complete generation.
Outcome:	The proposed explanations demonstrate faithfulness and efficiency of the proposed framework.

Copied to clipboard

WeiJie Liu, Yibin Zheng, Fang Kong

MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics (2026.acl-long)

Challenge:	Existing methods for table-to-text generation fail to capture the structure of tabular data or rely on complex attention mechanisms, limiting their applicability.
Approach:	They propose a question-driven self-supervised approach to enhance the model’s structural perception and representation capabilities by focusing on structure-related queries.
Outcome:	The proposed model improves its model's structural perception and representation capabilities by guiding it to capture local and global table structures.

Copied to clipboard

Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang, May Dongmei Wang

Breaking the Autoregressive Chain: Hyper-Parallel Decoding for Efficient LLM-Based Attribute Value Extraction (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable capabilities on general text, but their proficiency in specialized scientific domains remains uncharacterized.
Approach:	They evaluate the capabilities of large language models in metabolomics research using MetaBench . they found that models perform well on text generation tasks, but cross-database identifier grounding remains challenging .
Outcome:	The evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks.

Copied to clipboard

Theodore Glavas, Nikhita Vedula, Dushyanta Dhyani, Yilun Zhu, Shervin Malmasi